Eye hand task #236

resatomo · 2026-01-09T02:29:05Z

Sorry for a big PR.

Main changes are below,

Modified the task structure a little bit in the saccade-hand task, where subjects look at eye targets while changing their hand position
Added a new task about the saccade-reaching task, where subjects has to look at and reach the goal targets and initial position for eye and hand are independently controlled.
Changed the target shape for eye targets to the square
Added a new target position distributed at the side of the square for the eye calibration task
Added code in the online analysis for eye-hand task

leoscholl · 2026-01-16T19:44:19Z

analysis/online_analysis.py

+    This is for eye-related task that requires calibrated eye position
    '''

    def __init__(self, task_params, data_queue, calibration_dir='/var/tmp', buffer_time=1, ylim=1, px_per_cm=51.67, **kwargs):


can you remove the unnecessary variables calibration_dir='/var/tmp', buffer_time=1, ylim=1, px_per_cm=51.67, since they're not used

i think you can completely remove the init function here since it doesn't do anything anyway

leoscholl · 2026-01-16T19:48:46Z

analysis/online_analysis.py

+            # Update eye diameter
+            if self.calibrated_eye_pos.size > 2:
+                self.temp = np.array(values[0])[[0,1,4]]
+                self.eye_diam = np.roll(self.eye_diam, -1, axis=0)
+                self.eye_diam[-1] = self.temp


calibrated_eye_pos is never more than 2 elements, but we still get eye pos which contains diameter

Suggested change

# Update eye diameter

if self.calibrated_eye_pos.size > 2:

self.temp = np.array(values[0])[[0,1,4]]

self.eye_diam = np.roll(self.eye_diam, -1, axis=0)

self.eye_diam[-1] = self.temp

elif key == 'eye_pos':

# Update eye diameter

if self.eye_pos.size > 2:

self.temp = np.array(values[0])[[0,1,4]]

self.eye_diam = np.roll(self.eye_diam, -1, axis=0)

self.eye_diam[-1] = self.temp

leoscholl · 2026-01-16T19:55:54Z

analysis/online_analysis.py

        self.diam_plot.set_data(np.arange(len(self.eye_diam)) * 1/(int(self.task_params['fps'])) - self.buffer_time, 
                                self.eye_diam[:, 2]/self.px_per_cm)

+class EyeHandAnalysisWorker(SaccadeAnalysisWorker):


do you need to inherit SaccadeAnalysisWorker or can you inherit from BehaviorAnalysisWorker instead to simplify the inheritance?

leoscholl · 2026-01-16T19:58:57Z

built_in_tasks/target_capture_task.py

+    @staticmethod
+    def out_2D_square(nblocks=100, width=10, height=10, origin=(0,0,0)):
+        '''
+        Generates a sequence of 2D (x and z) targets at a point on the side of the square


Could you be a bit more descriptive? How many positions will there be? What exactly does it mean to be a point on the side of the square?

built_in_tasks/target_capture_task_eye.py

leoscholl · 2026-01-16T20:03:31Z

built_in_tasks/target_graphics.py

+        #self.cube.color = RED
+        #self.show()
+        pass

    def cue_trial_end_success(self):
        self.cube.color = GREEN

    def cue_trial_end_failure(self):
-        self.cube.color = YELLOW
-        self.hide()
+        self.cube.color = RED
+        #self.hide()

    def idle(self):
-        self.cube.color = RED
+        #self.cube.color = RED


i don't understand these changes, can you explain why you needed to do this? won't it affect other tasks?

It affects other tasks, but I changed this so that the cube target can change its color in the same way as the circular target.

can you remove the comments

leoscholl

i reviewed the changes to online_analysis.py
i had a hard time following the eye/hand tasks because they are so mixed up with the target capture task. i think it would be better if they were completely separate tasks. what do you think?

resatomo · 2026-01-16T21:35:35Z

i reviewed the changes to online_analysis.py i had a hard time following the eye/hand tasks because they are so mixed up with the target capture task. i think it would be better if they were completely separate tasks. what do you think?

I agree that separate tasks should be better for visuality and interpretability. I was just lazy because some states like wait and penalty are the same.

resatomo · 2026-01-22T01:59:39Z

I started to make this eye-hand task separate from the target capture task. But I noticed that many states overlapped between the 2 tasks. For example, the penalty states are almost similar. And also sometimes copying code itself is hard because I need to see both the screen target capture task and the target capture task.

So how about making new code without inheritance just for important states such as wait, target, delay, and so on, so that it can get easier for people to understand state flows. I want to inherit other penalty states. I think this is a compromised solution. What do you think?

leoscholl · 2026-01-22T18:38:54Z

ok well don't worry about it then.
i did have another idea that we should discuss, whether it is possible to create a single state machine that would work for both the eye/hand and hand/eye tasks. then it would be a lot easier to create a block design experiment with both tasks. i think it can work if you have the effector (hand, eye or both) required to pass each test be variable.

leoscholl · 2026-01-26T19:18:27Z

built_in_tasks/target_capture_task_eye.py


            yield indices, targs

+class EyeConstrainedHandCapture(HandConstrainedEyeCapture):


add a description of the changes

leoscholl · 2026-01-26T19:19:49Z

built_in_tasks/manualcontrolmultitasks.py


+class EyeHandSequenceTask(ManualControlMixin, EyeHandSequenceCapture):
+    '''
+    Saccade task while holding different targets by hand


this isn't the correct description i don't think.

leoscholl · 2026-01-26T19:24:59Z

built_in_tasks/target_capture_task_eye.py

            # Target 1 and 2 are for saccade. Target 3 is for hand
-            target1 = VirtualCircularTarget(target_radius=self.fixation_radius, target_color=target_colors[self.eye_target_color])
-            target2 = VirtualCircularTarget(target_radius=self.fixation_radius, target_color=target_colors[self.eye_target_color])
+            target1 = VirtualRectangularTarget(target_width=self.fixation_radius, target_height=self.fixation_radius/2, target_color=target_colors[self.eye_target_color])


why is height = width/2

built_in_tasks/target_capture_task_eye.py

leoscholl · 2026-01-26T19:38:29Z

built_in_tasks/target_capture_task_eye.py

-        target_pos = np.delete(self.targs[self.target_index],1)
-        d_eye = np.linalg.norm(eye_pos - target_pos)
-        return (d_eye <= self.fixation_radius + self.fixation_radius_buffer) or self.pause
+        eye_d = np.linalg.norm(eye_pos - self.targs[self.target_index,[0,2]])


if you can be bothered, it would be better if self.calibrated_eye_pos was 3D rather than ignoring the Y component of the target. For VR experiments this might be important

leoscholl · 2026-01-26T19:39:48Z

built_in_tasks/target_capture_task_eye.py

+    Then they need to fixate the first eye target and make a saccade for the second eye target. 2 of chain_length is only tested.
    '''

    fixation_radius = traits.Float(2.5, desc="Distance from center that is considered a broken fixation")


update description: it's now the radius (width?) of the fixation target

leoscholl · 2026-01-26T19:40:36Z

built_in_tasks/target_capture_task_eye.py

-    eye_target_color = traits.OptionsList("white", *target_colors, desc="Color of the eye target", bmi3d_input_options=list(target_colors.keys()))
+    fixation_target_color = traits.OptionsList("fixation_color", *target_colors, desc="Color of the eye target under fixation state", bmi3d_input_options=list(target_colors.keys()))
+    eye_target_color = traits.OptionsList("eye_color", *target_colors, desc="Color of the eye target", bmi3d_input_options=list(target_colors.keys()))
    fixation_radius_buffer = traits.Float(.5, desc="additional radius for eye target")


update description: the sum of the fixation radius and the buffer is what determines the break of fixation

leoscholl · 2026-01-26T19:40:49Z

built_in_tasks/target_capture_task_eye.py

+    fixation_target_color = traits.OptionsList("fixation_color", *target_colors, desc="Color of the eye target under fixation state", bmi3d_input_options=list(target_colors.keys()))
+    eye_target_color = traits.OptionsList("eye_color", *target_colors, desc="Color of the eye target", bmi3d_input_options=list(target_colors.keys()))
    fixation_radius_buffer = traits.Float(.5, desc="additional radius for eye target")
+    fixation_time = traits.Float(.2, desc="additional radius for eye target")


update description

leoscholl · 2026-01-26T19:41:19Z

built_in_tasks/target_capture_task_eye.py

+    eye_target_color = traits.OptionsList("eye_color", *target_colors, desc="Color of the eye target", bmi3d_input_options=list(target_colors.keys()))
    fixation_radius_buffer = traits.Float(.5, desc="additional radius for eye target")
+    fixation_time = traits.Float(.2, desc="additional radius for eye target")
+    incorrect_target_radius_buffer = traits.Float(.5, desc="additional radius for eye target")


update description

leoscholl · 2026-01-26T19:42:52Z

built_in_tasks/target_capture_task_eye.py

+    fixation_time = traits.Float(.2, desc="additional radius for eye target")
+    incorrect_target_radius_buffer = traits.Float(.5, desc="additional radius for eye target")
+    incorrect_target_penalty_time = traits.Float(1, desc="Length of penalty time for acquiring an incorrect target")
+    exclude_parent_traits = ['hold_time']


you might want to add a hidden_traits list for the above that are not commonly used - at least eye_target_color and fixation_target_color i think

leoscholl · 2026-01-26T19:49:28Z

built_in_tasks/target_capture_task_eye.py

-
-        # Initialize fixation state
-        self.num_hold_state = 0
+        self.isfixation_state = False


this variable could be better named, it's not immediately clear what this means. maybe avoid using state since that's confusing with the state machine. i suggest fixation_acquired or is_fixated

leoscholl · 2026-01-26T19:55:25Z

built_in_tasks/target_capture_task_eye.py

-            target = self.targets[next_idx % 2]
-            target.move_to_position(self.targs[next_idx % 2])
+            target = self.targets[next_idx]
+            target.move_to_position(self.targs[next_idx] - self.offset_cube)


what's the purpose of the offset?

leoscholl · 2026-01-26T19:55:57Z

built_in_tasks/target_capture_task_eye.py

+    def _end_reward(self):
+        super()._end_reward()


leoscholl · 2026-01-26T19:56:54Z

built_in_tasks/target_capture_task_eye.py

+        instantiate_targets = kwargs.pop('instantiate_targets', True)
+        if instantiate_targets:
+
+            # Target 1 and 2 are for saccade. Target 3 is for hand


what about target 4?

leoscholl · 2026-01-26T19:59:06Z

built_in_tasks/target_capture_task_eye.py

+    '''
+
+    exclude_parent_traits = ['delay_time', 'rand_delay']
+    rand_delay1 = traits.Tuple((0.4, 0.7), desc="Delay interval for eye")


can you call these rand_delay_eye and rand_delay_hand instead of 1 and 2

leoscholl · 2026-01-26T20:05:04Z

built_in_tasks/target_capture_task_eye.py

+    )
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)


parent class already makes targets, i think you should use
super(ScreenTargetCapture, self).__init__(*args, **kwargs)

same applies to all your other tasks

leoscholl · 2026-01-26T20:24:01Z

built_in_tasks/target_capture_task_eye.py

 class ScreenTargetCapture_Saccade(ScreenTargetCapture):
    '''
    Center-out saccade task. The controller for the cursor position is eye position.
    Hand cursor is also visible. You should remove the hand cursor by setting cursor_radius to 0 as needed.


Suggested change

Hand cursor is also visible. You should remove the hand cursor by setting cursor_radius to 0 as needed.

Hand cursor is also visible. You should remove the hand cursor by plant_visible to False as needed.

leoscholl · 2026-01-26T20:27:48Z

built_in_tasks/target_graphics.py

-        self.show()
+        #self.cube.color = RED
+        #self.show()
+        pass


i don't think we use this function but it should be

self.cube.color = self.target_color self.show()

resatomo added 11 commits December 22, 2025 11:03

Modified saccade-hand task

ed61da3

Update on online analysis

45f6d59

Fixed a bug and added colors

17ef92d

Added saccade reaching task

6bb54f8

Added color for eye task

3768605

Fixed event timing

4fa1410

Fixed event

7f254d1

fixed sync bugs

a38d03c

fix bug in transparent_delay

c312a13

Added different color for eye task

ff55956

added target position at the square for eye calib

399289a

resatomo assigned leoscholl Jan 9, 2026

Added incorrect_target_penalty n saccade reaching

7e824bd

leoscholl reviewed Jan 16, 2026

View reviewed changes

built_in_tasks/target_capture_task_eye.py Show resolved Hide resolved

leoscholl reviewed Jan 16, 2026

View reviewed changes

built_in_tasks/target_capture_task_eye.py Show resolved Hide resolved

leoscholl reviewed Jan 16, 2026

View reviewed changes

leoscholl requested changes Jan 16, 2026

View reviewed changes

resatomo added 2 commits January 21, 2026 15:57

Added eye-hand sequence task

6290c18

Fixed visualization of targets

a85ae58

resatomo added 2 commits January 21, 2026 18:06

Added buffer for incorrect target

bda6d9e

made eye cursor smaller in keyboard control

232d3f6

leoscholl reviewed Jan 26, 2026

View reviewed changes

built_in_tasks/target_capture_task_eye.py Show resolved Hide resolved

leoscholl reviewed Jan 26, 2026

View reviewed changes


		yield indices, targs

		class EyeConstrainedHandCapture(HandConstrainedEyeCapture):

	Hand cursor is also visible. You should remove the hand cursor by setting cursor_radius to 0 as needed.
	Hand cursor is also visible. You should remove the hand cursor by plant_visible to False as needed.

Eye hand task #236

Are you sure you want to change the base?

Eye hand task #236

Uh oh!

Conversation

resatomo commented Jan 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

resatomo Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leoscholl left a comment

Choose a reason for hiding this comment

Uh oh!

resatomo commented Jan 16, 2026

Uh oh!

resatomo commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leoscholl commented Jan 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leoscholl Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

resatomo Jan 16, 2026 •

edited

Loading

resatomo commented Jan 22, 2026 •

edited

Loading

leoscholl Jan 26, 2026 •

edited

Loading